Evaluating Listening Performance for COVID-19 Detection by Clinicians and Machine Learning: Comparative Study.

BACKGROUND: To date, performance comparisons between men and machines have been carried out in many health domains. Yet machine learning (ML) models and human performance comparisons in audio-based respiratory diagnosis remain largely unexplored. OBJECTIVE: The primary objective of this study was to compare human clinicians and an ML model in predicting COVID-19 from respiratory sound recordings. METHODS: In this study, we compared human clinicians and an ML model in predicting COVID-19 from respiratory sound recordings. Prediction performance on 24 audio samples (12 tested positive) made by 36 clinicians with experience in treating COVID-19 or other respiratory illnesses was compared with predictions made by an ML model trained on 1162 samples. Each sample consisted of voice, cough, and breathing sound recordings from 1 subject, and the length of each sample was around 20 seconds. We also investigated whether combining the predictions of the model and human experts could further enhance the performance in terms of both accuracy and confidence. RESULTS: The ML model outperformed the clinicians, yielding a sensitivity of 0.75 and a specificity of 0.83, whereas the best performance achieved by the clinicians was 0.67 in terms of sensitivity and 0.75 in terms of specificity. Integrating the clinicians' and the model's predictions, however, could enhance performance further, achieving a sensitivity of 0.83 and a specificity of 0.92. CONCLUSIONS: Our findings suggest that the clinicians and the ML model could make better clinical decisions via a cooperative approach and achieve higher confidence in audio-based respiratory diagnosis.

A summary of the ComParE COVID-19 challenges.

The COVID-19 pandemic has caused massive humanitarian and economic damage. Teams of scientists from a broad range of disciplines have searched for methods to help governments and communities combat the disease. One avenue from the machine learning field which has been explored is the prospect of a digital mass test which can detect COVID-19 from infected individuals' respiratory sounds. We present a summary of the results from the INTERSPEECH 2021 Computational Paralinguistics Challenges: COVID-19 Cough, (CCS) and COVID-19 Speech, (CSS).

Exploring Longitudinal Cough, Breath, and Voice Data for COVID-19 Progression Prediction via Sequential Deep Learning: Model Development and Validation.

BACKGROUND: Recent work has shown the potential of using audio data (eg, cough, breathing, and voice) in the screening for COVID-19. However, these approaches only focus on one-off detection and detect the infection, given the current audio sample, but do not monitor disease progression in COVID-19. Limited exploration has been put forward to continuously monitor COVID-19 progression, especially recovery, through longitudinal audio data. Tracking disease progression characteristics and patterns of recovery could bring insights and lead to more timely treatment or treatment adjustment, as well as better resource management in health care systems. OBJECTIVE: The primary objective of this study is to explore the potential of longitudinal audio samples over time for COVID-19 progression prediction and, especially, recovery trend prediction using sequential deep learning techniques. METHODS: Crowdsourced respiratory audio data, including breathing, cough, and voice samples, from 212 individuals over 5-385 days were analyzed, alongside their self-reported COVID-19 test results. We developed and validated a deep learning-enabled tracking tool using gated recurrent units (GRUs) to detect COVID-19 progression by exploring the audio dynamics of the individuals' historical audio biomarkers. The investigation comprised 2 parts: (1) COVID-19 detection in terms of positive and negative (healthy) tests using sequential audio signals, which was primarily assessed in terms of the area under the receiver operating characteristic curve (AUROC), sensitivity, and specificity, with 95% CIs, and (2) longitudinal disease progression prediction over time in terms of probability of positive tests, which was evaluated using the correlation between the predicted probability trajectory and self-reported labels. RESULTS: We first explored the benefits of capturing longitudinal dynamics of audio biomarkers for COVID-19 detection. The strong performance, yielding an AUROC of 0.79, a sensitivity of 0.75, and a specificity of 0.71 supported the effectiveness of the approach compared to methods that do not leverage longitudinal dynamics. We further examined the predicted disease progression trajectory, which displayed high consistency with longitudinal test results with a correlation of 0.75 in the test cohort and 0.86 in a subset of the test cohort with 12 (57.1%) of 21 COVID-19-positive participants who reported disease recovery. Our findings suggest that monitoring COVID-19 evolution via longitudinal audio data has potential in the tracking of individuals' disease progression and recovery. CONCLUSIONS: An audio-based COVID-19 progression monitoring system was developed using deep learning techniques, with strong performance showing high consistency between the predicted trajectory and the test results over time, especially for recovery trend predictions. This has good potential in the postpeak and postpandemic era that can help guide medical treatment and optimize hospital resource allocations. The changes in longitudinal audio samples, referred to as audio dynamics, are associated with COVID-19 progression; thus, modeling the audio dynamics can potentially capture the underlying disease progression process and further aid COVID-19 progression prediction. This framework provides a flexible, affordable, and timely tool for COVID-19 tracking, and more importantly, it also provides a proof of concept of how telemonitoring could be applicable to respiratory diseases monitoring, in general.

Sounds of COVID-19: exploring realistic performance of audio-based digital testing.

To identify Coronavirus disease (COVID-19) cases efficiently, affordably, and at scale, recent work has shown how audio (including cough, breathing and voice) based approaches can be used for testing. However, there is a lack of exploration of how biases and methodological decisions impact these tools' performance in practice. In this paper, we explore the realistic performance of audio-based digital testing of COVID-19. To investigate this, we collected a large crowdsourced respiratory audio dataset through a mobile app, alongside symptoms and COVID-19 test results. Within the collected dataset, we selected 5240 samples from 2478 English-speaking participants and split them into participant-independent sets for model development and validation. In addition to controlling the language, we also balanced demographics for model training to avoid potential acoustic bias. We used these audio samples to construct an audio-based COVID-19 prediction model. The unbiased model took features extracted from breathing, coughs and voice signals as predictors and yielded an AUC-ROC of 0.71 (95% CI: 0.65-0.77). We further explored several scenarios with different types of unbalanced data distributions to demonstrate how biases and participant splits affect the performance. With these different, but less appropriate, evaluation strategies, the performance could be overestimated, reaching an AUC up to 0.90 (95% CI: 0.85-0.95) in some circumstances. We found that an unrealistic experimental setting can result in misleading, sometimes over-optimistic, performance. Instead, we reported complete and reliable results on crowd-sourced data, which would allow medical professionals and policy makers to accurately assess the value of this technology and facilitate its deployment.

The feasibility of the PAM intervention to support treatment-adherence in people with hypertension in primary care: a randomised clinical controlled trial.

The PAM intervention is a behavioural intervention to support adherence to anti-hypertensive medications and therefore to lower blood pressure. This feasibility trial recruited 101 nonadherent patients (54% male, mean age 65.8 years) with hypertension and high blood pressure from nine general practices in the UK. The trial had 15.5% uptake and 7.9% attrition rate. Patients were randomly allocated to two groups: the intervention group (n = 61) received the PAM intervention as an adjunct to usual care; the control group (n = 40) received usual care only. At 3 months, biochemically validated medication adherence was improved by 20% (95% CI 3-36%) in the intervention than control, and systolic blood pressure was reduced by 9.16 mmHg (95% CI 5.69-12.64) in intervention than control. Improvements in medication adherence and reductions in blood pressure suggested potential intervention effectiveness. For a subsample of patients, improvements in medication adherence and reductions in full lipid profile (cholesterol 1.39 mmol/mol 95% CI 0.64-1.40) and in glycated haemoglobin (3.08 mmol/mol, 95% CI 0.42-5.73) favoured the intervention. A larger trial will obtain rigorous evidence about the potential clinical effectiveness and cost-effectiveness of the intervention.Trial registration Trial date of first registration 28/01/2019. ISRCTN74504989. https://doi.org/10.1186/ISRCTN74504989 .

Correction to: Assessing the acceptability of a text messaging service and smartphone app to support patient adherence to medications prescribed for high blood pressure: a pilot study.

[This corrects the article DOI: 10.1186/s40814-020-00666-2.].

Assessing the acceptability of a text messaging service and smartphone app to support patient adherence to medications prescribed for high blood pressure: a pilot study.

AIMS AND OBJECTIVES: This paper describes a pilot non-randomised controlled study of a highly tailored 56-day text messaging and smartphone app prototype intervention to increase adherence to anti-hypertensive medication in primary care. The aim of this study was to evaluate the acceptability of the intervention and obtain patients' views about the intervention content, the delivery mode, and the mechanisms by which the intervention supported medication adherence. METHODS: Patients diagnosed with hypertension were invited and recruited to the study via general practice text messages and attended a face to face meeting with a member of the researcher team. Participants were asked to test the text messaging intervention for 28 consecutive days and switch to the smartphone app for 28 more days. Participants completed baseline and follow-up questionnaires and took part in semi-structured telephone interviews. Digital log files captured patients' engagement with the intervention. Participant transcripts were analysed using thematic analysis. Descriptive statistics were used to summarise data from questionnaires and log files. A mixed methods analysis generated data to respond to the research questions. RESULTS: Seventy-nine patients expressed interest to participate in this study, of whom 23 (64% male, 82% above 60 years old) were registered to take part. With one drop-out, 22 participants tested the text messaging delivery mode (with 20 being interviewed) and four of them (17%) switched to the app (with 3 being interviewed). All participants engaged and interacted with the text messages and app notifications, and all participants found the intervention content and delivery mode acceptable. They also self-reported that the interactive elements of the intervention motivated them to take their medications as prescribed. CONCLUSION: This study provides evidence that the digital intervention is acceptable by hypertensive patients recruited in primary care. Future research could usefully investigate its feasibility and effectiveness using rigorous research methods. TRIAL REGISTRATION: ISRCTN12805654.